{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "# 简介\n",
    "本文使用PaddleHub模型库中的`deeplabv3p_xception65_humanseg`预训练模型，对健身和电影视频里的人物进行抠图，然后融合在一个宇宙纪录片的背景中，生成gif动画，或者附上配乐或配音，实现“太空广播操”和“宇宙演讲台”的创意。\n",
    "\n",
    "## 关于本项目\n",
    "> 针对项目还存在的改进空间、更有趣的Paddlehub玩法，希望大家多交流观点、介绍经验，共同学习进步。[个人主页](https://aistudio.baidu.com/aistudio/personalcenter/thirdview/90149)\n",
    "\n",
    "## 演示效果\n",
    "![file](https://ai-studio-static-online.cdn.bcebos.com/fa5f09d3d89c41c29f6279bcef7f52c43004363f430a4fc98727b861833ca19a)\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<video src=\"videos/out_gym_final.mp4\" controls  >\n",
       "      Your browser does not support the <code>video</code> element.\n",
       "    </video>"
      ],
      "text/plain": [
       "<IPython.core.display.Video object>"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import IPython\n",
    "IPython.display.Video('videos/out_gym_final.mp4')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "# 视频和音频的基本概念\n",
    "## 视频\n",
    "视频其实就是一系列的图像在时间维度上的排列，每一个视频帧就是一张图像。\n",
    "[百科链接](https://baike.baidu.com/item/%E8%A7%86%E9%A2%91%E5%B8%A7%E7%8E%87)\n",
    "> 每秒的帧数(fps)或者说帧率表示图形处理器处理场时每秒钟能够更新的次数。高的帧率可以得到更流畅、更逼真的动画。一般来说30fps就是可以接受的，但是将性能提升至60fps则可以明显提升交互感和逼真感，但是一般来说超过75fps一般就不容易察觉到有明显的流畅度提升了。如果帧率超过屏幕刷新率只会浪费图形处理的能力，因为监视器不能以这么快的速度更新，这样超过刷新率的帧率就浪费掉了。\n",
    "\n",
    "下图显示了一个视频文件的基本信息：\n",
    "![file](https://ai-studio-static-online.cdn.bcebos.com/851ac0abe182442790a4c40646e0bf766341cff042a740e2a0d27b9029d263e0)\n",
    "\n",
    "## 比特率\n",
    "[百科链接](https://baike.baidu.com/item/%E6%AF%94%E7%89%B9%E7%8E%87/1022775)\n",
    "> 比特率是指每秒传送的比特(bit)数。单位为 bps(Bit Per Second)，比特率越高，每秒传送数据就越多，画质就越清晰。声音中的比特率是指将模拟声音信号转换成数字声音信号后，单位时间内的二进制数据量，是间接衡量音频质量的一个指标。 视频中的比特率（码率）原理与声音中的相同，都是指由模拟信号转换为数字信号后，单位时间内的二进制数据量。\n",
    "\n",
    "## 音频采样率\n",
    "[百科链接](https://baike.baidu.com/item/%E9%9F%B3%E9%A2%91%E9%87%87%E6%A0%B7%E7%8E%87/9023551)\n",
    "> 音频采样率是指录音设备在一秒钟内对声音信号的采样次数，采样频率越高声音的还原就越真实越自然。在当今的主流采集卡上，采样频率一般共分为11025Hz、22050Hz、24000Hz、44100Hz、48000Hz五个等级，11025Hz能达到AM调幅广播的声音品质，而22050Hz和24000HZ能达到FM调频广播的声音品质，44100Hz则是理论上的CD音质界限，48000Hz则更加精确一些。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "# 加载工具包"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2020-06-01 21:45:02,682-INFO: font search path ['/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/mpl-data/fonts/ttf', '/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/mpl-data/fonts/afm', '/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/mpl-data/fonts/pdfcorefonts']\n",
      "2020-06-01 21:45:03,079-INFO: generated new fontManager\n",
      "Downloading deeplabv3p_xception65_humanseg\n",
      "[==================================================] 100.00%\n",
      "Uncompress /home/aistudio/.paddlehub/tmp/tmpwbco8dd8/deeplabv3p_xception65_humanseg\n",
      "[==================================================] 100.00%\n",
      "Successfully installed deeplabv3p_xception65_humanseg-1.0.0\n"
     ]
    }
   ],
   "source": [
    "# 下载预训练模型\r\n",
    "!hub install deeplabv3p_xception65_humanseg"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import paddlehub as hub #如果首次使用需要先安装飞桨框架和工具库，具体方法可以查看百度飞桨官网，非常简单\r\n",
    "from PIL import Image\r\n",
    "import os\r\n",
    "import cv2\r\n",
    "from tqdm import tqdm"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# 创建存放图片的文件夹\n",
    "!mkdir images\n",
    "!mkdir images/gym\n",
    "!mkdir images/earth\n",
    "!mkdir images/The_Matrix\n",
    "!mkdir images/gymseg_out\n",
    "!mkdir images/gymmerge_out\n",
    "!mkdir images/speechseg_out\n",
    "!mkdir images/speechmerge_out"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# 创建存放音视频的文件夹\r\n",
    "!mkdir videos"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "# 截取每一帧的视频\n",
    "> 视频数据与图像数据非常类似，都是由像素点组成的数据。在视频数据在非音频部分基本上可以视为多帧（张）图像数据的拼接，即三维图像的组合。由于视频数据与图像数据的相似性，在视频领域任务中大都可以借助图像方法来完成。\n",
    "\n",
    "从视频中抽帧有很多种方法，本文Python OpenCV库和FFmpeg命令行两种方法。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "## 使用OpenCV库抽取视频帧\n",
    "`a5.mp4`是一个健身视频片段，`data/data38147/earth.mp4`文件则是太空纪录片的一个小片段，下面演示如何使用Python OpenCV库进行视频抽帧。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# 定义一个抽帧函数\n",
    "def cut_frames(INPUT_FILE, OUTPUT_PATH, start_frame, end_frame):\n",
    "    reader = cv2.VideoCapture(INPUT_FILE)\n",
    "    # 视频帧的宽度\n",
    "    width = int(reader.get(cv2.CAP_PROP_FRAME_WIDTH))\n",
    "    # 视频帧的高度\n",
    "    height = int(reader.get(cv2.CAP_PROP_FRAME_HEIGHT))\n",
    "    # 确认是否读取到视频流  \n",
    "    print(reader.isOpened())\n",
    "    have_more_frame = True\n",
    "    c = 0\n",
    "    while have_more_frame:\n",
    "        have_more_frame, frame = reader.read()\n",
    "        c += 1\n",
    "        if c>= start_frame and c<= end_frame:\n",
    "            cv2.imwrite('%s%d.png'%(OUTPUT_PATH,c), frame)\n",
    "        if c>end_frame:\n",
    "            print('completely!')\n",
    "            break\n",
    "    \n",
    "    reader.release()\n",
    "    cv2.destroyAllWindows()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "True\n",
      "completely!\n"
     ]
    }
   ],
   "source": [
    "# 输入文件\r\n",
    "# INPUT_FILE = 'a5.mp4'\r\n",
    "# 输出图片位置\r\n",
    "# OUTPUT_PATH = 'images/gym/'\r\n",
    "# 开始视频帧\r\n",
    "# start_frame = 0\r\n",
    "# 结束视频帧\r\n",
    "# end_frame = 480\r\n",
    "# 开始抽帧\r\n",
    "cut_frames('a5.mp4', 'images/gym/', 0, 480)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "True\n",
      "completely!\n"
     ]
    }
   ],
   "source": [
    "# 太空背景抽帧\r\n",
    "cut_frames('data/data38147/earth.mp4', 'images/earth/', 0, 480)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "## 使用FFmpeg抽帧\n",
    "[FFmpeg](http://ffmpeg.org/documentation.html)是一套音视频多媒体处理开源框架，它提供了对音视频的采集、编码、解码、转码、音视频分离、合并、流化、过滤器等丰富的功能。目前，业界的视频播放主要有三种架构：MPC， MPlayer和VLC，占据市场90%的份额。而三个架构均使用或者融合FFmpeg的视频解码技术。FFmpeg堪称多媒体领域的神器。\n",
    "在FFmpeg中，视频抽帧只需要一行命令：\n",
    "```bash\n",
    "ffmpeg -i 输入视频.mp4 -r 30 输出目录/%d.png\n",
    "```\n",
    "具体的可选参数：\n",
    "- -r 指定抽取的帧率，即从视频中每秒钟抽取图片的数量。1代表每秒抽取一帧。\n",
    "- -t，表示取t秒时间的帧\n",
    "- -ss，表示截取帧初始时间\n",
    "- -vframes，表示截取多少帧"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ffmpeg version 2.8.15-0ubuntu0.16.04.1 Copyright (c) 2000-2018 the FFmpeg developers\n",
      "  built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.10) 20160609\n",
      "  configuration: --prefix=/usr --extra-version=0ubuntu0.16.04.1 --build-suffix=-ffmpeg --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --cc=cc --cxx=g++ --enable-gpl --enable-shared --disable-stripping --disable-decoder=libopenjpeg --disable-decoder=libschroedinger --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librtmp --enable-libschroedinger --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxvid --enable-libzvbi --enable-openal --enable-opengl --enable-x11grab --enable-libdc1394 --enable-libiec61883 --enable-libzmq --enable-frei0r --enable-libx264 --enable-libopencv\n",
      "  libavutil      54. 31.100 / 54. 31.100\n",
      "  libavcodec     56. 60.100 / 56. 60.100\n",
      "  libavformat    56. 40.101 / 56. 40.101\n",
      "  libavdevice    56.  4.100 / 56.  4.100\n",
      "  libavfilter     5. 40.101 /  5. 40.101\n",
      "  libavresample   2.  1.  0 /  2.  1.  0\n",
      "  libswscale      3.  1.101 /  3.  1.101\n",
      "  libswresample   1.  2.101 /  1.  2.101\n",
      "  libpostproc    53.  3.100 / 53.  3.100\n",
      "Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'data/data38254/hkdg.mp4':\n",
      "  Metadata:\n",
      "    major_brand     : isom\n",
      "    minor_version   : 512\n",
      "    compatible_brands: isomiso2avc1mp41\n",
      "    encoder         : Lavf58.29.100\n",
      "  Duration: 00:01:13.40, start: 0.023220, bitrate: 2872 kb/s\n",
      "    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt470bg/unknown/unknown), 1280x720, 2727 kb/s, 30 fps, 30 tbr, 15360 tbn, 60 tbc (default)\n",
      "    Metadata:\n",
      "      handler_name    : VideoHandler\n",
      "    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 157 kb/s (default)\n",
      "    Metadata:\n",
      "      handler_name    : SoundHandler\n",
      "Output #0, image2, to 'images/The_Matrix/%d.png':\n",
      "  Metadata:\n",
      "    major_brand     : isom\n",
      "    minor_version   : 512\n",
      "    compatible_brands: isomiso2avc1mp41\n",
      "    encoder         : Lavf56.40.101\n",
      "    Stream #0:0(und): Video: png, rgb24, 1280x720, q=2-31, 200 kb/s, 30 fps, 30 tbn, 30 tbc (default)\n",
      "    Metadata:\n",
      "      handler_name    : VideoHandler\n",
      "      encoder         : Lavc56.60.100 png\n",
      "Stream mapping:\n",
      "  Stream #0:0 -> #0:0 (h264 (native) -> png (native))\n",
      "Press [q] to stop, [?] for help\n",
      "frame=  480 fps=111 q=-0.0 Lsize=N/A time=00:00:16.00 bitrate=N/A dup=1 drop=0    \n",
      "video:293244kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown\n"
     ]
    }
   ],
   "source": [
    "# -r 指定抽取的帧率，即从视频中每秒钟抽取图片的数量。1代表每秒抽取一帧。\r\n",
    "# !ffmpeg -i data/data38254/hkdg.mp4 -r 30 images/The_Matrix/%d.png\r\n",
    "# 截取前480帧\r\n",
    "!ffmpeg -i data/data38254/hkdg.mp4 -r 30 -vframes 480 images/The_Matrix/%d.png"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "# 人像抠图"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import os\r\n",
    "def humanseg(images,output_dir):  \r\n",
    "    #装载预训练模型\r\n",
    "    module = hub.Module(name=\"deeplabv3p_xception65_humanseg\")\r\n",
    "    result = module.segmentation(data={\"image\":images}, output_dir=output_dir)\r\n",
    "    # print (result)\r\n",
    "    return result\r\n",
    "\r\n",
    "\r\n",
    "def file_list(listdir):\r\n",
    "    '''\r\n",
    "    读取listdir文件夹里的序号图片文件，将名字形成im_list\r\n",
    "    '''\r\n",
    "    im_list=[]\r\n",
    "    files = os.listdir(listdir)\r\n",
    "    for i in range(len(files)) :\r\n",
    "        im_list.append('%s%d.png' %(listdir,i+1) )\r\n",
    "    return im_list"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# 健身照片批量抠图\r\n",
    "humanseg(file_list('images/gym/'),'images/gymseg_out')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# 演讲照片批量抠图\r\n",
    "humanseg(file_list('images/The_Matrix/'),'images/speechseg_out')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "# 视频融合\n",
    "这里参考[PaddleHub创意赛：当维密天使来到东北](https://aistudio.baidu.com/aistudio/projectdetail/446851)项目的做法，利用PIL库的alpha_composite将两组图片融合逐一融合"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "## 使用FFmpeg合成gif\n",
    "### 去掉视频的logo\n",
    "语法：`-vf delogo=x:y:w:h[:t[:show]]`\n",
    "- `x:y` 离左上角的坐标\n",
    "- `w:h` logo的宽和高\n",
    "- `t:` 矩形边缘的厚度默认值4\n",
    "- `show` 若设置为1有一个绿色的矩形，默认值0。\n",
    "### 截取时间设置\n",
    "- `-ss` 开始时间\n",
    "- `-t` 总共截取时间\n",
    "- `-r` 每秒帧数\n",
    "### 缩小输出\n",
    "语法： `-vf scale=w:h`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ffmpeg version 2.8.15-0ubuntu0.16.04.1 Copyright (c) 2000-2018 the FFmpeg developers\n",
      "  built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.10) 20160609\n",
      "  configuration: --prefix=/usr --extra-version=0ubuntu0.16.04.1 --build-suffix=-ffmpeg --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --cc=cc --cxx=g++ --enable-gpl --enable-shared --disable-stripping --disable-decoder=libopenjpeg --disable-decoder=libschroedinger --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librtmp --enable-libschroedinger --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxvid --enable-libzvbi --enable-openal --enable-opengl --enable-x11grab --enable-libdc1394 --enable-libiec61883 --enable-libzmq --enable-frei0r --enable-libx264 --enable-libopencv\n",
      "  libavutil      54. 31.100 / 54. 31.100\n",
      "  libavcodec     56. 60.100 / 56. 60.100\n",
      "  libavformat    56. 40.101 / 56. 40.101\n",
      "  libavdevice    56.  4.100 / 56.  4.100\n",
      "  libavfilter     5. 40.101 /  5. 40.101\n",
      "  libavresample   2.  1.  0 /  2.  1.  0\n",
      "  libswscale      3.  1.101 /  3.  1.101\n",
      "  libswresample   1.  2.101 /  1.  2.101\n",
      "  libpostproc    53.  3.100 / 53.  3.100\n",
      "Input #0, image2, from 'images/gymmerge_out/%d.png':\n",
      "  Duration: 00:00:19.20, start: 0.000000, bitrate: N/A\n",
      "    Stream #0:0: Video: png, rgba(pc), 1920x1080, 25 fps, 25 tbr, 25 tbn, 25 tbc\n",
      "\u001b[0;33mNo pixel format specified, yuv444p for H.264 encoding chosen.\n",
      "Use -pix_fmt yuv420p for compatibility with outdated media players.\n",
      "\u001b[0m\u001b[1;36m[libx264 @ 0x130faa0] \u001b[0musing cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 AVX2 LZCNT BMI2\n",
      "\u001b[1;36m[libx264 @ 0x130faa0] \u001b[0mprofile High 4:4:4 Predictive, level 4.0, 4:4:4 8-bit\n",
      "\u001b[1;36m[libx264 @ 0x130faa0] \u001b[0m264 - core 148 r2643 5c65704 - H.264/MPEG-4 AVC codec - Copyleft 2003-2015 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=4 threads=34 lookahead_threads=5 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=15 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00\n",
      "Output #0, mp4, to 'out_gym.mp4':\n",
      "  Metadata:\n",
      "    encoder         : Lavf56.40.101\n",
      "    Stream #0:0: Video: h264 (libx264) ([33][0][0][0] / 0x0021), yuv444p, 1920x1080, q=-1--1, 15 fps, 15360 tbn, 15 tbc\n",
      "    Metadata:\n",
      "      encoder         : Lavc56.60.100 libx264\n",
      "Stream mapping:\n",
      "  Stream #0:0 -> #0:0 (png (native) -> h264 (libx264))\n",
      "Press [q] to stop, [?] for help\n",
      "frame=   30 fps=9.7 q=-1.0 Lsize=     497kB time=00:00:01.86 bitrate=2182.9kbits/s    \n",
      "video:496kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.235758%\n",
      "\u001b[1;36m[libx264 @ 0x130faa0] \u001b[0mframe I:1     Avg QP:17.71  size:152513\n",
      "\u001b[1;36m[libx264 @ 0x130faa0] \u001b[0mframe P:8     Avg QP:19.96  size: 28411\n",
      "\u001b[1;36m[libx264 @ 0x130faa0] \u001b[0mframe B:21    Avg QP:23.28  size:  6079\n",
      "\u001b[1;36m[libx264 @ 0x130faa0] \u001b[0mconsecutive B-frames:  6.7%  0.0%  0.0% 93.3%\n",
      "\u001b[1;36m[libx264 @ 0x130faa0] \u001b[0mmb I  I16..4:  2.5% 88.8%  8.7%\n",
      "\u001b[1;36m[libx264 @ 0x130faa0] \u001b[0mmb P  I16..4:  0.6%  3.6%  1.0%  P16..4: 25.0% 11.8%  7.0%  0.0%  0.0%    skip:51.0%\n",
      "\u001b[1;36m[libx264 @ 0x130faa0] \u001b[0mmb B  I16..4:  0.1%  0.3%  0.0%  B16..8: 20.9%  2.3%  0.6%  direct: 0.7%  skip:75.1%  L0:41.1% L1:47.3% BI:11.6%\n",
      "\u001b[1;36m[libx264 @ 0x130faa0] \u001b[0m8x8 transform intra:82.7% inter:73.0%\n",
      "\u001b[1;36m[libx264 @ 0x130faa0] \u001b[0mcoded y,u,v intra: 57.6% 31.1% 26.9% inter: 7.0% 2.2% 2.1%\n",
      "\u001b[1;36m[libx264 @ 0x130faa0] \u001b[0mi16 v,h,dc,p: 25% 32% 14% 28%\n",
      "\u001b[1;36m[libx264 @ 0x130faa0] \u001b[0mi8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 11% 16% 39%  6%  4%  3%  6%  5% 10%\n",
      "\u001b[1;36m[libx264 @ 0x130faa0] \u001b[0mi4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 29% 18%  8%  7%  7%  6%  8%  7% 10%\n",
      "\u001b[1;36m[libx264 @ 0x130faa0] \u001b[0mWeighted P-Frames: Y:0.0% UV:0.0%\n",
      "\u001b[1;36m[libx264 @ 0x130faa0] \u001b[0mref P L0: 85.1%  4.0%  9.2%  1.6%\n",
      "\u001b[1;36m[libx264 @ 0x130faa0] \u001b[0mref B L0: 90.3%  9.2%  0.5%\n",
      "\u001b[1;36m[libx264 @ 0x130faa0] \u001b[0mref B L1: 99.0%  1.0%\n",
      "\u001b[1;36m[libx264 @ 0x130faa0] \u001b[0mkb/s:2029.84\n"
     ]
    }
   ],
   "source": [
    "# ffmpeg将图片合成为mp4，从第15秒开始截取2秒视频，去掉视频logo\n",
    "!ffmpeg -ss 00:00:15 -t 00:00:02 -r 15 -i images/gymmerge_out/%d.png -vf delogo=1650:0:220:120:100:0 out_gym.mp4"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ffmpeg version 2.8.15-0ubuntu0.16.04.1 Copyright (c) 2000-2018 the FFmpeg developers\n",
      "  built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.10) 20160609\n",
      "  configuration: --prefix=/usr --extra-version=0ubuntu0.16.04.1 --build-suffix=-ffmpeg --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --cc=cc --cxx=g++ --enable-gpl --enable-shared --disable-stripping --disable-decoder=libopenjpeg --disable-decoder=libschroedinger --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librtmp --enable-libschroedinger --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxvid --enable-libzvbi --enable-openal --enable-opengl --enable-x11grab --enable-libdc1394 --enable-libiec61883 --enable-libzmq --enable-frei0r --enable-libx264 --enable-libopencv\n",
      "  libavutil      54. 31.100 / 54. 31.100\n",
      "  libavcodec     56. 60.100 / 56. 60.100\n",
      "  libavformat    56. 40.101 / 56. 40.101\n",
      "  libavdevice    56.  4.100 / 56.  4.100\n",
      "  libavfilter     5. 40.101 /  5. 40.101\n",
      "  libavresample   2.  1.  0 /  2.  1.  0\n",
      "  libswscale      3.  1.101 /  3.  1.101\n",
      "  libswresample   1.  2.101 /  1.  2.101\n",
      "  libpostproc    53.  3.100 / 53.  3.100\n",
      "Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'out_gym.mp4':\n",
      "  Metadata:\n",
      "    major_brand     : isom\n",
      "    minor_version   : 512\n",
      "    compatible_brands: isomiso2avc1mp41\n",
      "    encoder         : Lavf56.40.101\n",
      "  Duration: 00:00:02.00, start: 0.000000, bitrate: 2037 kb/s\n",
      "    Stream #0:0(und): Video: h264 (High 4:4:4 Predictive) (avc1 / 0x31637661), yuv444p, 1920x1080, 2032 kb/s, 15 fps, 15 tbr, 15360 tbn, 30 tbc (default)\n",
      "    Metadata:\n",
      "      handler_name    : VideoHandler\n",
      "Output #0, gif, to 'out_gym.gif':\n",
      "  Metadata:\n",
      "    major_brand     : isom\n",
      "    minor_version   : 512\n",
      "    compatible_brands: isomiso2avc1mp41\n",
      "    encoder         : Lavf56.40.101\n",
      "    Stream #0:0(und): Video: gif, bgr8, 384x216, q=2-31, 200 kb/s, 15 fps, 100 tbn, 15 tbc (default)\n",
      "    Metadata:\n",
      "      handler_name    : VideoHandler\n",
      "      encoder         : Lavc56.60.100 gif\n",
      "Stream mapping:\n",
      "  Stream #0:0 -> #0:0 (h264 (native) -> gif (native))\n",
      "Press [q] to stop, [?] for help\n",
      "frame=   30 fps=0.0 q=-0.0 Lsize=     782kB time=00:00:02.00 bitrate=3204.3kbits/s    \n",
      "video:781kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.130120%\n"
     ]
    }
   ],
   "source": [
    "# 将mp4转换为gif，并将动图分辨率缩放到384*216\n",
    "!ffmpeg -i out_gym.mp4 -vf scale=384:216 out_gym.gif"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "## PIL库的alpha_composite图片融合\n",
    "- 将太空背景与健身者抠图一一融合\n",
    "- 将太空背景与演讲者抠图的前300张图片一一融合（演讲者视频只有前10秒有人物，300多张后的抠图是空白。）"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "1. 健身者"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def alpha_composite(fore_image,base_image,output_dir):\r\n",
    "    '''\r\n",
    "    将前景图片fore_image与背景图片base_image融合，输出到文件夹output_dir\r\n",
    "    '''\r\n",
    "    #利用RGBA的方式读取图片\r\n",
    "    base_image_op = Image.open(base_image).convert(\"RGBA\")\r\n",
    "    fore_image_op = Image.open(fore_image).convert(\"RGBA\")\r\n",
    "    # print (base_image_op)\r\n",
    "    # print (fore_image_op)\r\n",
    "\r\n",
    "    #调整照片的size，因为背景图片较小，所以我调整背景图片来适应前景图片\r\n",
    "    base_image_op = base_image_op.resize(fore_image_op.size)  \r\n",
    "    # base_image = base_image.rotate(90)  #如果图片角度不同，还可以旋转图片\r\n",
    "    \r\n",
    "    #将前景图片与背景图片融合\r\n",
    "    image_c = Image.alpha_composite(base_image_op, fore_image_op)\r\n",
    "    image_c.save(output_dir + fore_image.replace('images/gymseg_out/','') )\r\n",
    "\r\n",
    "    return image_c\r\n",
    "# 视频转化图片的文件夹\r\n",
    "files_v = os.listdir('images/earth/') \r\n",
    "#人像抠图的文件夹\r\n",
    "fore_path='images/gymseg_out/'\r\n",
    "#背景图片\r\n",
    "base_dir='images/earth/'\r\n",
    "#合成图片输出的文件夹\r\n",
    "output_dir='images/gymmerge_out/'\r\n",
    "\r\n",
    "for i in range(len(files_v)):\r\n",
    "    base_image='%s%d.png'%(base_dir,i+1)\r\n",
    "    fore_image='%s%d.png'%(fore_path,i+1)\r\n",
    "    alpha_composite(fore_image,base_image,output_dir)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "2. 演讲者"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def alpha_composite(fore_image,base_image,output_dir):\r\n",
    "    '''\r\n",
    "    将前景图片fore_image与背景图片base_image融合，输出到文件夹output_dir\r\n",
    "    '''\r\n",
    "    #利用RGBA的方式读取图片\r\n",
    "    base_image_op = Image.open(base_image).convert(\"RGBA\")\r\n",
    "    fore_image_op = Image.open(fore_image).convert(\"RGBA\")\r\n",
    "    # print (base_image_op)\r\n",
    "    # print (fore_image_op)\r\n",
    "\r\n",
    "    #调整照片的size，因为背景图片较小，所以我调整背景图片来适应前景图片\r\n",
    "    base_image_op = base_image_op.resize(fore_image_op.size)  \r\n",
    "    # base_image = base_image.rotate(90)  #如果图片角度不同，还可以旋转图片\r\n",
    "    \r\n",
    "    #将前景图片与背景图片融合\r\n",
    "    image_c = Image.alpha_composite(base_image_op, fore_image_op)\r\n",
    "    image_c.save(output_dir + fore_image.replace('images/speechseg_out/','') )\r\n",
    "\r\n",
    "    return image_c\r\n",
    "# 视频转化图片的文件夹\r\n",
    "files_v = os.listdir('images/earth/') \r\n",
    "#人像抠图的文件夹\r\n",
    "fore_path='images/speechseg_out/'\r\n",
    "#背景图片\r\n",
    "base_dir='images/earth/'\r\n",
    "#合成图片输出的文件夹\r\n",
    "output_dir='images/speechmerge_out/'\r\n",
    "\r\n",
    "for i in range(len(files_v)):\r\n",
    "    # 只融合前300张图片\r\n",
    "    if i <= 300:\r\n",
    "        base_image='%s%d.png'%(base_dir,i+1)\r\n",
    "        fore_image='%s%d.png'%(fore_path,i+1)\r\n",
    "        alpha_composite(fore_image,base_image,output_dir)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### 使用OpenCV库图片合成视频"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### 太空健身操"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# 视频转化图片的文件夹\r\n",
    "def png2video(image_dir,out_video):\r\n",
    "    '''\r\n",
    "    将文件夹image_dir中的每一帧图片合成视频out_video\r\n",
    "    '''\r\n",
    "    #fourcc代表视频流的合成格式\r\n",
    "    fourcc = cv2.VideoWriter_fourcc(*'XVID')\r\n",
    "    #以(1280,720)的分辨率，25帧/s的速度，fourcc的视频流格式来合成视频out_video\r\n",
    "    out = cv2.VideoWriter(out_video,fourcc, 25.0, (1280,720))\r\n",
    "    files = os.listdir(image_dir)\r\n",
    "\r\n",
    "    files_v = os.listdir('images/gymmerge_out/') \r\n",
    "\r\n",
    "    for i in range(len(files_v)): \r\n",
    "        image='%s%d.png' %(image_dir,i)\r\n",
    "        \r\n",
    "        try:\r\n",
    "            img = cv2.imread(image)\r\n",
    "            img = cv2.resize(img,(1280,720))\r\n",
    "            out.write(img)#保存帧\r\n",
    "\r\n",
    "        except:\r\n",
    "            continue\r\n",
    "\r\n",
    "    out.release() #关闭视频\r\n",
    "# 使用OpenCV库进行视频融合\r\n",
    "png2video('images/gymmerge_out/','videos/out_gym.mp4')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "#### 提取广播体操音频并合成\n",
    "- `9.mp3`是广播体操音频文件"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ffmpeg version 2.8.15-0ubuntu0.16.04.1 Copyright (c) 2000-2018 the FFmpeg developers\n",
      "  built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.10) 20160609\n",
      "  configuration: --prefix=/usr --extra-version=0ubuntu0.16.04.1 --build-suffix=-ffmpeg --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --cc=cc --cxx=g++ --enable-gpl --enable-shared --disable-stripping --disable-decoder=libopenjpeg --disable-decoder=libschroedinger --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librtmp --enable-libschroedinger --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxvid --enable-libzvbi --enable-openal --enable-opengl --enable-x11grab --enable-libdc1394 --enable-libiec61883 --enable-libzmq --enable-frei0r --enable-libx264 --enable-libopencv\n",
      "  libavutil      54. 31.100 / 54. 31.100\n",
      "  libavcodec     56. 60.100 / 56. 60.100\n",
      "  libavformat    56. 40.101 / 56. 40.101\n",
      "  libavdevice    56.  4.100 / 56.  4.100\n",
      "  libavfilter     5. 40.101 /  5. 40.101\n",
      "  libavresample   2.  1.  0 /  2.  1.  0\n",
      "  libswscale      3.  1.101 /  3.  1.101\n",
      "  libswresample   1.  2.101 /  1.  2.101\n",
      "  libpostproc    53.  3.100 / 53.  3.100\n",
      "Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'videos/out_gym.mp4':\n",
      "  Metadata:\n",
      "    major_brand     : isom\n",
      "    minor_version   : 512\n",
      "    compatible_brands: isomiso2mp41\n",
      "    encoder         : Lavf58.31.101\n",
      "  Duration: 00:00:19.16, start: 0.000000, bitrate: 5072 kb/s\n",
      "    Stream #0:0(und): Video: mpeg4 (Simple Profile) (mp4v / 0x7634706D), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], 5071 kb/s, 25 fps, 25 tbr, 12800 tbn, 25 tbc (default)\n",
      "    Metadata:\n",
      "      handler_name    : VideoHandler\n",
      "\u001b[1;36m[libx264 @ 0x1ba06e0] \u001b[0musing SAR=1/1\n",
      "\u001b[1;36m[libx264 @ 0x1ba06e0] \u001b[0musing cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 AVX2 LZCNT BMI2\n",
      "\u001b[1;36m[libx264 @ 0x1ba06e0] \u001b[0mprofile High, level 3.1\n",
      "\u001b[1;36m[libx264 @ 0x1ba06e0] \u001b[0m264 - core 148 r2643 5c65704 - H.264/MPEG-4 AVC codec - Copyleft 2003-2015 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=22 lookahead_threads=3 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00\n",
      "Output #0, mp4, to 'videos/out_gym_offlogo.mp4':\n",
      "  Metadata:\n",
      "    major_brand     : isom\n",
      "    minor_version   : 512\n",
      "    compatible_brands: isomiso2mp41\n",
      "    encoder         : Lavf56.40.101\n",
      "    Stream #0:0(und): Video: h264 (libx264) ([33][0][0][0] / 0x0021), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], q=-1--1, 25 fps, 12800 tbn, 25 tbc (default)\n",
      "    Metadata:\n",
      "      handler_name    : VideoHandler\n",
      "      encoder         : Lavc56.60.100 libx264\n",
      "Stream mapping:\n",
      "  Stream #0:0 -> #0:0 (mpeg4 (native) -> h264 (libx264))\n",
      "Press [q] to stop, [?] for help\n",
      "frame=  479 fps= 36 q=-1.0 Lsize=    4128kB time=00:00:19.08 bitrate=1772.2kbits/s    \n",
      "video:4123kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.108882%\n",
      "\u001b[1;36m[libx264 @ 0x1ba06e0] \u001b[0mframe I:2     Avg QP:18.26  size: 68857\n",
      "\u001b[1;36m[libx264 @ 0x1ba06e0] \u001b[0mframe P:344   Avg QP:21.39  size: 10788\n",
      "\u001b[1;36m[libx264 @ 0x1ba06e0] \u001b[0mframe B:133   Avg QP:24.90  size:  2801\n",
      "\u001b[1;36m[libx264 @ 0x1ba06e0] \u001b[0mconsecutive B-frames: 62.6%  0.0%  3.1% 34.2%\n",
      "\u001b[1;36m[libx264 @ 0x1ba06e0] \u001b[0mmb I  I16..4: 28.5% 67.0%  4.5%\n",
      "\u001b[1;36m[libx264 @ 0x1ba06e0] \u001b[0mmb P  I16..4:  0.2%  1.2%  0.1%  P16..4: 22.5% 11.6%  9.0%  0.0%  0.0%    skip:55.3%\n",
      "\u001b[1;36m[libx264 @ 0x1ba06e0] \u001b[0mmb B  I16..4:  0.1%  0.7%  0.0%  B16..8: 25.7%  2.5%  0.5%  direct: 0.6%  skip:69.8%  L0:40.3% L1:50.0% BI: 9.7%\n",
      "\u001b[1;36m[libx264 @ 0x1ba06e0] \u001b[0m8x8 transform intra:77.7% inter:65.8%\n",
      "\u001b[1;36m[libx264 @ 0x1ba06e0] \u001b[0mcoded y,uvDC,uvAC intra: 56.2% 60.2% 22.6% inter: 15.3% 12.2% 0.5%\n",
      "\u001b[1;36m[libx264 @ 0x1ba06e0] \u001b[0mi16 v,h,dc,p: 54% 18% 16% 12%\n",
      "\u001b[1;36m[libx264 @ 0x1ba06e0] \u001b[0mi8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 13% 22% 36%  4%  4%  4%  5%  4%  8%\n",
      "\u001b[1;36m[libx264 @ 0x1ba06e0] \u001b[0mi4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 17% 23% 15%  6%  8%  7% 10%  6%  8%\n",
      "\u001b[1;36m[libx264 @ 0x1ba06e0] \u001b[0mi8c dc,h,v,p: 52% 23% 18%  6%\n",
      "\u001b[1;36m[libx264 @ 0x1ba06e0] \u001b[0mWeighted P-Frames: Y:0.0% UV:0.0%\n",
      "\u001b[1;36m[libx264 @ 0x1ba06e0] \u001b[0mref P L0: 78.0%  9.4%  9.9%  2.7%\n",
      "\u001b[1;36m[libx264 @ 0x1ba06e0] \u001b[0mref B L0: 94.0%  4.2%  1.8%\n",
      "\u001b[1;36m[libx264 @ 0x1ba06e0] \u001b[0mref B L1: 98.9%  1.1%\n",
      "\u001b[1;36m[libx264 @ 0x1ba06e0] \u001b[0mkb/s:1762.55\n"
     ]
    }
   ],
   "source": [
    "# 提取音频\r\n",
    "# !ffmpeg -i 9.mp3 -ss 00:01:00 -t 00:00:18 -acodec copy videos/output9.mp3\r\n",
    "\r\n",
    "# 去掉logo\r\n",
    "!ffmpeg -i videos/out_gym.mp4 -vf delogo=1650:0:220:120:100:0 videos/out_gym_offlogo.mp4"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ffmpeg version 2.8.15-0ubuntu0.16.04.1 Copyright (c) 2000-2018 the FFmpeg developers\n",
      "  built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.10) 20160609\n",
      "  configuration: --prefix=/usr --extra-version=0ubuntu0.16.04.1 --build-suffix=-ffmpeg --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --cc=cc --cxx=g++ --enable-gpl --enable-shared --disable-stripping --disable-decoder=libopenjpeg --disable-decoder=libschroedinger --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librtmp --enable-libschroedinger --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxvid --enable-libzvbi --enable-openal --enable-opengl --enable-x11grab --enable-libdc1394 --enable-libiec61883 --enable-libzmq --enable-frei0r --enable-libx264 --enable-libopencv\n",
      "  libavutil      54. 31.100 / 54. 31.100\n",
      "  libavcodec     56. 60.100 / 56. 60.100\n",
      "  libavformat    56. 40.101 / 56. 40.101\n",
      "  libavdevice    56.  4.100 / 56.  4.100\n",
      "  libavfilter     5. 40.101 /  5. 40.101\n",
      "  libavresample   2.  1.  0 /  2.  1.  0\n",
      "  libswscale      3.  1.101 /  3.  1.101\n",
      "  libswresample   1.  2.101 /  1.  2.101\n",
      "  libpostproc    53.  3.100 / 53.  3.100\n",
      "Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'videos/out_gym_offlogo.mp4':\n",
      "  Metadata:\n",
      "    major_brand     : isom\n",
      "    minor_version   : 512\n",
      "    compatible_brands: isomiso2avc1mp41\n",
      "    encoder         : Lavf56.40.101\n",
      "  Duration: 00:00:19.16, start: 0.000000, bitrate: 1764 kb/s\n",
      "    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], 1762 kb/s, 25 fps, 25 tbr, 12800 tbn, 50 tbc (default)\n",
      "    Metadata:\n",
      "      handler_name    : VideoHandler\n",
      "\u001b[0;35m[mp3 @ 0x1b9f280] \u001b[0mSkipping 0 bytes of junk at 561.\n",
      "Input #1, mp3, from 'videos/output9.mp3':\n",
      "  Metadata:\n",
      "    major_brand     : isom\n",
      "    minor_version   : 512\n",
      "    compatible_brands: isomiso2mp41\n",
      "    encoder         : Lavf56.40.101\n",
      "  Duration: 00:00:18.00, start: 0.011995, bitrate: 128 kb/s\n",
      "    Stream #1:0: Audio: mp3, 44100 Hz, stereo, s16p, 128 kb/s\n",
      "    Metadata:\n",
      "      encoder         : Lavf\n",
      "\u001b[1;35m[mp4 @ 0x1ba0e40] \u001b[0m\u001b[0;33mCodec for stream 0 does not use global headers but container format requires global headers\n",
      "\u001b[0m\u001b[1;35m[mp4 @ 0x1ba0e40] \u001b[0m\u001b[0;33mCodec for stream 1 does not use global headers but container format requires global headers\n",
      "\u001b[0mOutput #0, mp4, to 'videos/out_gym_final.mp4':\n",
      "  Metadata:\n",
      "    major_brand     : isom\n",
      "    minor_version   : 512\n",
      "    compatible_brands: isomiso2avc1mp41\n",
      "    encoder         : Lavf56.40.101\n",
      "    Stream #0:0(und): Video: h264 ([33][0][0][0] / 0x0021), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 1762 kb/s, 25 fps, 25 tbr, 12800 tbn, 12800 tbc (default)\n",
      "    Metadata:\n",
      "      handler_name    : VideoHandler\n",
      "    Stream #0:1: Audio: mp3 (i[0][0][0] / 0x0069), 44100 Hz, stereo, 128 kb/s\n",
      "    Metadata:\n",
      "      encoder         : Lavf\n",
      "Stream mapping:\n",
      "  Stream #0:0 -> #0:0 (copy)\n",
      "  Stream #1:0 -> #0:1 (copy)\n",
      "Press [q] to stop, [?] for help\n",
      "frame=  479 fps=0.0 q=-1.0 Lsize=    4420kB time=00:00:19.04 bitrate=1901.8kbits/s    \n",
      "video:4123kB audio:281kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.363859%\n"
     ]
    }
   ],
   "source": [
    "# 添加音频合成视频\n",
    "!ffmpeg -y -i videos/out_gym_offlogo.mp4 -i videos/output9.mp3 -vcodec copy -acodec copy videos/out_gym_final.mp4"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### 宇宙演讲者"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# 视频转化图片的文件夹\r\n",
    "def png2video(image_dir,out_video):\r\n",
    "    '''\r\n",
    "    将文件夹image_dir中的每一帧图片合成视频out_video\r\n",
    "    '''\r\n",
    "    #fourcc代表视频流的合成格式\r\n",
    "    fourcc = cv2.VideoWriter_fourcc(*'XVID')\r\n",
    "    #以(1280,720)的分辨率，25帧/s的速度，fourcc的视频流格式来合成视频out_video\r\n",
    "    out = cv2.VideoWriter(out_video,fourcc, 25.0, (1280,720))\r\n",
    "    files = os.listdir(image_dir)   \r\n",
    "    # 视频转化图片的文件夹\r\n",
    "    files_v = os.listdir('images/speechmerge_out/') \r\n",
    "\r\n",
    "    for i in range(len(files_v)): \r\n",
    "        image='%s%d.png' %(image_dir,i)\r\n",
    "        \r\n",
    "        try:\r\n",
    "            img = cv2.imread(image)\r\n",
    "            img = cv2.resize(img,(1280,720))\r\n",
    "            out.write(img)#保存帧\r\n",
    "\r\n",
    "        except:\r\n",
    "            continue\r\n",
    "\r\n",
    "    out.release() #关闭视频\r\n",
    "# 使用OpenCV库进行视频融合\r\n",
    "png2video('images/speechmerge_out/','videos/out_speech.mp4')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "#### 合成音频"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ffmpeg version 2.8.15-0ubuntu0.16.04.1 Copyright (c) 2000-2018 the FFmpeg developers\n",
      "  built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.10) 20160609\n",
      "  configuration: --prefix=/usr --extra-version=0ubuntu0.16.04.1 --build-suffix=-ffmpeg --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --cc=cc --cxx=g++ --enable-gpl --enable-shared --disable-stripping --disable-decoder=libopenjpeg --disable-decoder=libschroedinger --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librtmp --enable-libschroedinger --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxvid --enable-libzvbi --enable-openal --enable-opengl --enable-x11grab --enable-libdc1394 --enable-libiec61883 --enable-libzmq --enable-frei0r --enable-libx264 --enable-libopencv\n",
      "  libavutil      54. 31.100 / 54. 31.100\n",
      "  libavcodec     56. 60.100 / 56. 60.100\n",
      "  libavformat    56. 40.101 / 56. 40.101\n",
      "  libavdevice    56.  4.100 / 56.  4.100\n",
      "  libavfilter     5. 40.101 /  5. 40.101\n",
      "  libavresample   2.  1.  0 /  2.  1.  0\n",
      "  libswscale      3.  1.101 /  3.  1.101\n",
      "  libswresample   1.  2.101 /  1.  2.101\n",
      "  libpostproc    53.  3.100 / 53.  3.100\n",
      "Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'data/data38254/hkdg.mp4':\n",
      "  Metadata:\n",
      "    major_brand     : isom\n",
      "    minor_version   : 512\n",
      "    compatible_brands: isomiso2avc1mp41\n",
      "    encoder         : Lavf58.29.100\n",
      "  Duration: 00:01:13.40, start: 0.023220, bitrate: 2872 kb/s\n",
      "    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt470bg/unknown/unknown), 1280x720, 2727 kb/s, 30 fps, 30 tbr, 15360 tbn, 60 tbc (default)\n",
      "    Metadata:\n",
      "      handler_name    : VideoHandler\n",
      "    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 157 kb/s (default)\n",
      "    Metadata:\n",
      "      handler_name    : SoundHandler\n",
      "Output #0, mp3, to 'videos/speech.mp3':\n",
      "  Metadata:\n",
      "    major_brand     : isom\n",
      "    minor_version   : 512\n",
      "    compatible_brands: isomiso2avc1mp41\n",
      "    TSSE            : Lavf56.40.101\n",
      "    Stream #0:0(und): Audio: mp3 (libmp3lame), 44100 Hz, stereo, fltp (default)\n",
      "    Metadata:\n",
      "      handler_name    : SoundHandler\n",
      "      encoder         : Lavc56.60.100 libmp3lame\n",
      "Stream mapping:\n",
      "  Stream #0:1 -> #0:0 (aac (native) -> mp3 (libmp3lame))\n",
      "Press [q] to stop, [?] for help\n",
      "size=     189kB time=00:00:12.01 bitrate= 128.5kbits/s    \n",
      "video:0kB audio:188kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.184763%\n"
     ]
    }
   ],
   "source": [
    "# 分离原视频的音频流，注意，这里只提取12秒的音频\r\n",
    "!ffmpeg -i data/data38254/hkdg.mp4 -ss 00:00:00 -t 00:00:12 videos/speech.mp3 "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# 添加音频合成视频\r\n",
    "!ffmpeg -i videos/out_speech.mp4 -i videos/speech.mp3  -vcodec copy -acodec copy videos/out_speech_final.mp4 -loglevel quiet"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "# 静态效果演示"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "![file](https://ai-studio-static-online.cdn.bcebos.com/d3d8807ab70c4f4680acad6155bbf464bb58291078e3440588800b36adc159dd)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "![file](https://ai-studio-static-online.cdn.bcebos.com/bdf0f5916dad4f67b490c8579b42cff499d7b3f7947e4b689ba63c714ada89c1)\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "PaddlePaddle 1.7.2 (Python 3.5)",
   "language": "python",
   "name": "py35-paddle1.2.0"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
